1 Impedance Control is Selectively Tuned to 2 Multiple Directions of Movement 3 4 5

25 Humans are able to learn tool-handling tasks, such as carving, demonstrating 26 their competency to make movements in unstable environments with varied 27 directions. When faced with a single direction of instability, humans learn to 28 selectively co-contract their arm muscles tuning the mechanical stiffness of the 29 limb endpoint to stabilize movements. This study examines, for the first time, 30 subjects simultaneously adapting to two distinct directions of instability -a 31 situation that may typically occur when using tools. Subjects learned to perform 32 reaching movements in two directions, each of which had lateral instability 33 requiring control of impedance. The subjects were able to adapt to these 34 unstable interactions and switch between movements in the two directions; they 35 did so by learning to selectively control the endpoint stiffness counteracting the 36 environmental instability without superfluous stiffness in other directions. This 37 finding demonstrates that the CNS can simultaneously tune the mechanical 38 impedance of the limbs to multiple movements by learning movement specific 39 solutions. Furthermore, it suggests that the impedance controller learns as a 40 function of the state of the arm rather than a general strategy. 41 42 43 INTRODUCTION 44 45 Many common tasks requiring tools such as chiseling or drilling are unstable. 46 This instability amplifies the effects of motor noise (Hamilton et al. 2004; Harris 47 and Wolpert 1998) and can lead to unpredictable and unsuccessful action 48 (Burdet et al. 2006). For instance, the hand of an apprentice carpenter may slip 49 when chiseling a piece of rough wood. However, with practice he will learn to 50 control movements with suitable force and impedance in all directions and carve 51 skillfully. In order to develop these skills he will need to tune his limb impedance 52 (Burdet et al. 2001; Gomi and Kawato 1996; Hogan 1984, 1985) to the 53 instabilities which may vary across the workspace. In order to modulate the limb 54 impedance he can either change the limb posture (Hogan 1985; Rancourt and 55 Hogan 2001; Trumbower et al. 2009), modulate feedback control (Krutky et al. 56 2010; Loram et al. 2011; Loram et al. 2009; Morasso 2011) or co-activate 57 muscles (Damm and McIntyre 2008; Hogan 1984; Milner 2002). In an unstable or 58 unpredictable environment, the central nervous system (CNS) learns to co59 contract suitable muscle pairs involved in the movement (Burdet et al. 2001; 60 Franklin et al. 2007a; Franklin et al. 2003b) stabilizing the limb endpoint and 61 minimizing the production and effect of motor noise (Selen et al. 2005; Selen et 62 al. 2009). However, all previous studies investigating impedance control in 63 reaching movements examined only a single movement direction (Burdet et al. 64 2001; Burdet et al. 2006; Franklin et al. 2003a; Franklin et al. 2008; Franklin et al. 65 2007a; Franklin et al. 2003b; Franklin et al. 2007b; Osu et al. 2003; Osu et al. 66 2002; Takahashi et al. 2001; Wong et al. 2009a, b). Can humans learn 67 impedance control models to compensate for different directions of instability 68 across the workspace? 69 70 Adaptation to stable dynamics takes place in the joint coordinates of the 71 neuromuscular system (Shadmehr and Mussa-Ivaldi 1994) and generalizes 72 across the workspace (Conditt et al. 1997; Gandolfo et al. 1996; Malfait et al. 73 2005; Malfait et al. 2002; Shadmehr and Moussavi 2000; Shadmehr and Mussa74 Ivaldi 1994). When different dynamics or visuomotor transformations are learned 75 one after another, interference between the two learned models results 76 (Brashers-Krug et al. 1996; Caithness et al. 2004; Karniel and Mussa-Ivaldi 2002; 77 Mattar and Ostry 2007; Osu et al. 2004; Shadmehr and Brashers-Krug 1997) 78 making it difficult to learn independent internal models. Exceptions have only 79 been found for sufficiently different environments (Krakauer et al. 1999), for 80 random presentations (Osu et al. 2004), and in bimanual movements where the 81 context allows independent learning of multiple models (Howard et al. 2008, 82 2010; Nozaki et al. 2006). However, when different force directions are learned 83 as a function of different movement directions, no interference occurs (Shadmehr 84 and Mussa-Ivaldi 1994; Thoroughman and Taylor 2005) as a single model of the 85 overall environment is learned. 86 87 If impedance control uses similar neural structures as learning stable dynamics, it 88 should be possible to independently control limb stiffness across the workspace. 89 To examine this hypothesis, subjects reached to two different targets with 90 instability applied orthogonally to each trajectory (lateral instability). We 91 examined whether endpoint stiffness adaptation occurred in the same manner 92 whether subjects moved to a single target with one direction of instability or 93 randomly to one of two targets, each with its own instability direction. If only a 94 single co-contraction model is learned, then the stiffness in the multiple 95 movement direction condition would not be optimal for both directions. However, 96 if the impedance controller can learn a single model generalizing across the 97 workspace or switches between multiple models for each direction of movement, 98 then subjects may learn the optimal stiffness for adaptation to each direction. 99 100 101 MATERIALS AND METHODS 102 103 Simulation of general and selective endpoint stiffness. 104 Simulations were performed to examine how the CNS may adapt endpoint 105 stiffness of the arm when lateral instability is applied on the hand, during planar 106 movements in different directions (Fig. 1A). The simulated task (identical to the 107 experimental task) consisted of performing two 25 cm long point-to-point 108 movements: from (0, 31) cm relative to the shoulder to (0, 56) cm for the D1 109 movement and from the same start to (14, 52) cm for the D2 movement (which is 110 separated from D1 by 35° clockwise). The simulations explored possible 111 strategies for the CNS to move successfully in the lateral instability caused by a 112 divergent force field (DF), orthogonal to the movement trajectory (Fig 1A), 113 defined by: 114 0 F x F ζ ⊥ ⊥     =          [1] 115 where F⊥ and F indicate the force components normal and parallel to the 116 straight line from start to end points respectively, ζ = 300 N/m, and x⊥ is the 117 lateral deviation of the hand from this straight line. The endpoint stiffness 118 necessary to compensate for this instability was simulated at the middle of the 119 movement. 120 121 A simplified version of the muscle limb model from (Selen et al. 2009) was used 122 to predict both the baseline null field (NF) stiffness and theoretical adaptations to 123 the instability. The force attributed to each muscle (Fm) was solved for such that 124 the minimal summed muscle activation was produced that could both result in the 125 required endpoint force and sufficient endpoint stiffness in the direction 126 perpendicular to the movement direction. Six muscles were simulated: two single 127 joint shoulder muscles, two single joint elbow muscles and two biarticular 128 muscles. Muscle forces were required to produce the joint torques (T) which 129 created the appropriate endpoint forces F for each direction of movement: 130 ( ) ( ) 1 1 T T T m − − = = F J T J D F [2] 131 where J is the Jacobian matrix of the limb configuration, D is a 6 x 2 matrix of the 132 moment arms of the muscles around the shoulder and elbow joints. The endpoint 133 stiffness (K) for a given muscle activation pattern was found using: 134 ( ) ( ) 1 1 T T T base m m d diag c dθ − −   = + −     J K J R D F D F J [3] 135 where cm is the stiffness scaling constant. As in (Selen et al. 2009) we used cm = 136 75 m for all muscles, which was based on published joint torque stiffness 137 (Franklin and Milner 2003; Gomi and Osu 1998) and muscle force stiffness 138 (Edman and Josephson 2007) regressions. The matrix Rbase is a 2 x 2 matrix 139 [3.18 2.15; 2.34 6.18] Nm/rad containing the baseline joint stiffness found 140 experimentally (Gomi and Osu 1998) when the muscle activation is zero. 141 142 In order to examine the predicted changes in endpoint stiffness, the baseline 143 value of endpoint stiffness of the limb for null field movements needs to be 144 determined. Previous studies have demonstrated that the null field endpoint 145 stiffness is usually around 200 N/m in the direction perpendicular to that of the 146 movement (Franklin et al. 2004), which may be due to interaction between 147 stiffness and movement variability (Burdet et al. 2001; Burdet et al. 2006; Lametti 148 et al. 2007). The simulated endpoint stiffness for each movement direction was 149 solved subject to the constraint that the endpoint stiffness perpendicular to the 150 movement was larger than 200 N/m. This produced stiffness ellipses qualitatively 151 similar to those previously measured (Franklin et al. 2003a; Franklin et al. 2004) 152 which we used for the baseline stiffness. For the predictions in the unstable force 153 fields, the stiffness constraint was such that the endpoint stiffness perpendicular 154 to the movement was larger than 500 N/m. This produced endpoint stiffness 155 which compensated for the lateral instability (-300 N/m) such that the net 156 endpoint stiffness was maintained (Burdet et al. 2001; Franklin et al. 2004). The 157 optimal muscle activation pattern (Fm) producing no change in endpoint force 158 was solved for each prediction using the fmincon function (Matlab 2007a) on 159 equation 3. Additional constraints are specifically detailed for each prediction 160 below. 161 162 In order to adapt movements in two directions simultaneously, one possible 163 strategy for the CNS is to learn to co-contract all antagonistic pairs of muscles 164 equally such that movements in both directions would be stabilized. To simulate 165 this global co-contraction strategy, a single co-contraction term, which increased 166 the activation of all muscles, was found such that the stiffness perpendicular to 167 the movement direction was sufficient to compensate for the instability of the 168 force field for both movement directions. Specifically, the increase in each 169 element of Fm was constrained to be equal (ΔF1=ΔF2=ΔF3=ΔF4=ΔF5=ΔF6) and 170 occurred for adaptation to both directions of movement (ΔFm in D1 = ΔFm in D2). 171 172 Alternatively the CNS may be able to learn to selectively control the endpoint 173 stiffness to counteract the effect of lateral instability as in previous work (Burdet 174 et al. 2001), but only able to learn a single endpoint stiffness or set of muscle 175 activations. To simulate this strategy, we solved for the optimal set of muscle 176 activations which would produce the required endpoint stiffness for both 177 movement directions simultaneously. Specifically, the same increase in each 178 element of Fm occurred for adaptation to both directions of movement (ΔFm in D1 179 = ΔFm in D2). 180 181 Finally, the possibility is that the CNS adapts endpoint stiffness optimally to the 182 instability specific to each direction of movement. This would mean that either the 183 CNS could switch from one learned impedance model to another dependent on 184 the movement state or learns an impedance model which can be modulated 185 across the workspace. To simulate this strategy, the optimal muscle activations 186 were solved independently for each movement direction. Specifically, no 187 constraint was applied on the change in muscle force; such that independent 188 patterns could be learned for each movement direction (ΔFm in D1 was not 189 constrained to be equal to ΔFm in D2). 190 191 Experimental Methods. 192 Ten right-handed male subjects without any known neurological problem aged 193 between 19 and 34 years old participated in the study. The institutional ethics 194 committee approved the experiments and subjects provided informed consent. 195 196 Subjects sat on an adjustable chair with harness over their upper trunk, which 197 prevented movement of the trunk (Fig 1D). A subject specific custom-molded 198 thermoplastic cuff was used to restrict motion of the wrist and firmly attach the 199 subject’s hand and forearm to the manipulandum (Fig 1E). The forearm and cuff 200 were coupled to the handle of the parallel-link direct drive air magnet floating 201 manipulandum (PFM) (Gomi and Kawato 1996; Gomi and Kawato 1997). The 202 coupling of the forearm to the manipulandum was such that measurements could 203 be made and movements could be performed without subjects explicitly grasping 204 the handle. Detailed figures of the PFM and the experimental setup are found in 205 (Franklin et al. 2007b). The arm was restricted to planar motion of the shoulder 206 and elbow, where the positive x and positive y directions correspond to the right 207 and forward of the subject, respectively. Hand position was estimated from the 208 PFM joint encoders (409,600 pulse/rev), and force exerted on/by the hand was 209 measured using a force sensor (resolution 0.006 N) placed between the handle 210 and the manipulandum’s links. Both force and position signals were sampled at 211 500Hz. 212 213 Subjects were instructed to perform point-to-point movements from a starting 214 circle (1.5 cm diameter) centered at (0, 31) cm relative to the shoulder and 215 towards two targets denoted by D1 and D2 respectively within 600±100 ms. Each 216 target was a 2.5 cm diameter circle that was 25 cm apart from the starting point. 217 D1 movements were performed to a target at (0, 56) cm straight-ahead of the 218 shoulder, and D2 movements towards a circle centered at (14, 52) cm, which 219 was at 35o clockwise rotation from the first target. A cursor representing the 220 actual hand position was beamed from a ceiling mounted projector onto an 221 opaque cover which prevented subjects from seeing their hands or the robot. 222 Start circle and the selected target circle were also displayed before and during 223 each trial in order to indicate the movement subjects needed to make. A screen 224 in front of subjects displayed feedback about successful and non-successful 225 movements after each trial. An unsuccessful trial was indicated by “out of target”, 226 “too slow” or “too fast” on a monitor placed in front of subject after the trial. All 227 movements were recorded during the experiment, whether successful or not. 228 There was no time constraint as to when the following trial should start and so 229 subjects could rest between trials. Subjects were required to bring the cursor 230 inside the start circle before a beep sound signaled start of the movement. Each 231 time the cursor was brought within the start circle, a trial was initiated by three 232 beeps at 500 ms intervals. Subjects were instructed to start moving their hand at 233 the third beep and reach the specified target circle by the fourth beep, 600 ms 234 later. Finally, two subsequent beeps, 500ms apart, indicated that the trial had 235 finished. 236